A distributed diffusion method for dynamic load balancing on parallel computers

Abstract. Parallel application can be divided into tasks that can be executed simultaneously. A mechanism for assigning these tasks to the processors is required. The objective is to minimize the overall execution time of a single application running in parallel on a multicomputer system. We propose a new dynamic load balancing algorithm based on the diffusion approach which employs overlapping balancing domains (a processor and its neighbors) to achieve global balancing. Since current diffusion methods consider discrete units, the algorithms may produce solutions which, although they are locally balanced, prove to be globally unbalanced. Our method solves this problem taking into account the load maximum difference between two processors within each domain, providing a more efficient load balancing process. This method is performed in a distributed fashion and can easily be scaled to support highly parallel machines. The algorithm has been applied to different interconnection networks and the results obtained are very encouraging